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ABSTRACT 

Secret-key agreement protocols over wiretap channels con- 
trolled by a state parameter are studied. The entire state se- 
quence is known (non-causally) to the sender but not to the 
receiver and the eavesdropper. Upper and lower bounds on 
the secret-key capacity are established both with and with- 
out public discussion. The proposed coding scheme involves 
constructing a codebook to create common reconstruction of 
the state sequence at the sender and the receiver and another 
secret-key codebook constructed by random binning. For the 
special case of Gaussian channels, with no public discussion, 
— the secret-key generation with dirty paper problem, the 
gap between our bounds is at-most 1/2 bit and the bounds co- 
incide in the high signal-to-noise ratio and high interference- 
to-noise ratio regimes. In the presence of public discussion 
our bounds coincide, yielding the capacity, when then the 
channels of the receiver and the eavesdropper satisfy an in- 
dependent noise condition. 

1. INTRODUCTION 

Many applications in cryptography require that the legitimate 
terminals have shared secret keys, not available to unautho- 
rized parties. Information theoretic security encompasses the 
study of source and channel coding techniques to generate 
secret-keys between legitimate terminals. In the channel cod- 
ing literature, an early work in this area is the wiretap chan- 
nel model III . It consists of three terminals one sender, one 
receiver and one eavesdropper The sender communicates to 
the receiver and the eavesdropper over a discrete-memoryless 
broadcast channel. A notion of equivocation-rate the nor- 
malized conditional entropy of the transmitted message given 
the observation at the eavesdropper, is introduced, and the 
tradeoff between information rate and equivocation rate is 
studied. Perfect secrecy capacity, defined as the maximum 
information rate under the constraint that the equivocation 
rate approaches the information rate asymptotically in the 
block length is of particular interest. Information transmit- 
ted at this rate can be naturally used as a shared secret-key 
between the sender and the receiver. In the source coding 
setup ||2l O the two terminals observe correlated source se- 
quences and use a public discussion channel for communica- 
tion. Any information sent over this channel is available to an 
eavesdropper The terminals generate a common secret-key 
that is concealed from the eavesdropper in the same sense as 
the wiretap channel the equivocation 

In the present paper we consider a secret-key agreement 
problem when the sender and the receiver communicate over 
a channel controlled by a state parameter The state parame- 
ter is known to the sender but not to the receiver or the eaves- 



dropper. The problem of transmitting information on such 
channels, without secrecy constraints, is studied in A 
random binning strategy is proposed and shown to achieve 
the capacity. Costa |5| studies the problem of communicat- 
ing over an additive noise Gaussian channel with an addi- 
tive interference sequence known to the transmitter and es- 
tablishes that there is no loss in capacity if the interference 
sequence is known only to the transmitter. In the present 
work, we study the problem of generating a common secret 
key between the sender and the receiver over such channels. 
Our proposed coding scheme is not based on the Gel'fand- 
Pinsker binning technique for sending an information mes- 
sage over such channels. Instead our codebook is designed to 
create a common reconstruction sequence at the sender and 
the receiver and distilling a secret-key based on this common 
sequence. 

In related works, the problem of secret-message trans- 
mission over wiretap channels controlled by a state parame- 
ter is studied in ||6l|7l. In these works an achievable coding 
scheme is proposed that combines Gel'fand Pinsker coding 
and coding for the wiretap channel. As discussed earlier, 
our coding scheme is based on a different approach and in 
general yields higher achievable rates. A related problem of 
common reconstruction of state sequences has been studied 
recently in JS] |9l . The problem of secret-key agreement with 
symmetric channel state information at the sender and the 
legitimate receiver has been studied in jJOl. However the 
coding scheme involved is based on the fact that the termi- 
nals have knowledge of the common state sequence to be- 
gin with. After this paper was submitted, we learnt about a 
recent work ifTTl where the problem of communicating over 
over channels with non-causal CSI is used as a building block 
for characterizing the tradeoff between secret-key and secret- 
message transmission. 

2. PROBLEM SETUP 

As Fig. [T] illustrates, the channel model has three terminals 
— a sender, a receiver and an eavesdropper The sender 
communicates with the other two terminals over a discrete- 
memoryless-channel with transition probability Py^.y^\x,s{-) 
where x denotes the channel input symbol, whereas 34- and 
ye denote the channel output symbols at the receiver and the 
eavesdropper respectively. The symbol s denotes a state vari- 
able that controls the channel transition probability. We as- 
sume that it is sampled i.i.d. from a distribution ps in each 
channel use. Further, the entire sequence s" is known to the 
sender before the communication begins. 

In defining the secret-key capacity we separately con- 
sider the cases when a public discussion channel is and is 
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Figure 1 : Wiretap channel controlled by a state parameter The channel transition probability Py^^y^jjr^j () is controlled by a state 
parameter s. The entire source sequence s" is known to the sender but not to the receiver or the eavesdropper. The sender and 
receiver generate a secret key K at the end of the transmission. 



not present. 

2.1 No Public Discussion 

A length n encoder is defined as follows. The sender sam- 
ples a random variables u from the conditional distribution 
Pu\s" i'Wr)- The encoding function produces a channel input 
sequence x" ^ f„{u,s") and transmits it over n uses of the 
channel. At time ; the symbol x, is transmitted and the legit- 
imate receiver and the eavesdropper observe output symbols 
Yri and yd respectively, sampled from the conditional distri- 
bution Py^^y^\x,s{')- The sender and receiver compute secret 
keys K = g„{u,s") and / = hn{y"). A rate R is achievable 
if there exists a sequence of encoding functions such that 
for some sequence e„ that vanishes as n — > ex., we have that 
Pr(K- ^ l)<e„ and ^H{k) >R-e„ and 

-I{K;y^)<E„. (1) 

The largest achievable rate is the secret-key capacity. 

2.2 Presence of Public Discussion 

When a public discussion channel is present, the described 
protocol follows closely the interactive communication pro- 
tocol in [3J. The sender transmits symbols xj , . . . ,x„ at times 
< i'l < /2 < . . . < in over the wiretap channel. At these 
times the receiver and the eavesdropper observe symbols 
y,i,...,y,-„ and yei,---,yen respectively. In the remaining 
times the sender and receiver exchange messages y/, and 
where 1 < t < k. For convenience we let ;„+i = k + I. The 
eavesdropper observes both i//, and (j)f. 

More specifically the sender and receiver sample random 
variables u and v from conditional distributions Pu\s"{'\s") 
and Pv{-) and observe that v is independent of (u,s"). 
• At times < f < /i, the sender generates 

(j)t = <t>,(u,s", 1/'^') and the receiver generates 

V/, = These messages are exchanged 

over the public channel. 



• At times ij , 1 < j < «, the sender generates x/ = 
Xj{u,s" and sends it over the channel. The re- 
ceiver and eavesdropper observe yrj ad y^ j respectively. 
For these times we set i^,^ = (pij = 0. 

• For times ij < t < ij+i, where 1 < j < n, the sender 
and receiver compute 0, = <t>((u,s", \//'^') and i/, = 
1',(\/,y/, 0'^' ) respectively and exchange them over the 
public channel. 

• At time k+l, the sender and receiver compute k = 
g„{u,s" jXj/'^) and the receiver computes / ^ h„{v,y" ,^^). 
We require that for some sequence e„ that vanishes as 

n — )• oo, Pr{K^ I) < E„ and 

-/(^c;y^^V/^(/)')<e„. (2) 
n 

The secret-key rate is defined as ^H{k) and the largest 
achievable secret-key rate is the capacity. 

3. MAIN RESULTS 

Our main results are upper and lower bounds on the secret- 
key capacity, which coincide in some special cases. We again 
consider the cases of no public discussion and public discus- 
sion separately. 

3.1 No Public Discussion 

We first provide an achievable rate (lower bound) on the 
secret-key capacity. 

Theorem 1 An achievable secret-key rate without public 
discussion is 

R = max /(u;yr)-/(u;ye), (3) 

Pu^Px\s.u 

where the maximization is over all auxiliary random vari- 
ables u that satisfy the Markov condition u — !• (x,s) — > 
(yr,ye) and furthermore satisfy the constraint that 

I{u-yr)-I{u;s)>Q. (4) 



3.2 With Public Discussion 



The intuition behind the coding scheme is as follows. 
Upon observing s", the sender communicates the best pos- 
sible reproduction u" of the state sequence to the receiver 
Now both the sender and the receiver observe a common se- 
quence u". The set of all codewords u" is binned into 2"^ 
bins and the bin-index is declared to be the secret key. 

We note that the lower bound can be easily extended to 
the case of two-sided CSI. If the receiver observes another 
state sequence s,-, correlated with s according to a joint distri- 
bution Ps.sri', ■) then the achievable rate expression (O holds 
provided that we augment the received symbol by (yi-, s,-). 

Finally for the case of symmetric CSI i.e., when = s, 
the constraint (HJi is redundant as clearly I{u;yi-,s) ~I{u; s) > 
holds. Furthermore the resulting achievable rate, 

R = max /(u;yr,s)-/(u;ye) 

Pu,Px\s.u 

is indeed the secret-key capacity as established in our earlier 
work IfTOl . 

Finally we note that the problem of secret-key agreement 
is different from the secret-message transmission problem 
considered in ll6l|T2l|2l. This is because the secret-key can 
be an arbitrary function of the state sequence (known only to 
the transmitter) whereas the secret-message needs to be inde- 
pendent function of the state sequence. For comparison, the 
best known lower bound on the secret-message transmission 
problem is stated below. 

Proposition 1 ^ [72] [7|/ An achievable secret message rate 
for wiretap channel with non-causal transmiter CSI is 



R < max 

Pu^Pxlu.s 



/(u;yr)-max (/(u;s),/(u;ye)) . 



(5) 



We note that whenever the maximizing u satisfies, 
/(u;ye) > /(u;s) > /(u;ye), the secret-key rate ([3]) is strictly 
better than the secret-message rate 

The following theorem develops an upper bound on 
secret-key capacity that is amenable to numerical computa- 
tion. 

Theorem 2 The secret-key capacity in absence of public dis- 
cussion is upper bounded by C < R^, where 



R^ ^ min max/(x, s;yr|ye), 



(6) 



where !^ denotes all the joint distributions p* , that have 
the same marginal distribution as the original channel. 

The intuition behind the upper bound is as follows. We 
create a degraded channel by revealing the output of the 
eavesdropper to the legitimate receiver. We further assume 
a channel with two inputs (x",s") i.e., the state sequence s" 
is not arbitrary, but rather a part of the input codeword with 
distribution ps,.. The secrecy capacity of the resulting wiretap 
channel is then given by /(x, s;yr|ye). 

Our proposed upper and lower bounds coincide, yielding 
capacity in some special cases. We present one such case in 
section [ 



In this section we provide lower and upper bounds on the 
secret-key capacity with public discussion. We first provide 
a lower bound below. 

Theorem 3 An achievable secret-key rate with public dis- 
cussion is: 



^disc ~ 



max max/(x,s;yr) 

Pxls 



(7) 



where R is the lower bound attained without public discus- 
sion in Theorem\l\ 

The achievability scheme involves a natural modification 
of Maurer's coding scheme E] |2l to incorporate the pres- 
ence of the state parameter and involves a single round of 
discussion. In particular, the sender generate a sequence x" 
according to the conditional distribution p^isiA^) trans- 
mits over n channel uses. At the end of the transmission, 
the receiver sends the bin index of y", so that the sender can 
recover this sequence given (x",s"). 

Next we provide an upper bound on the secret-key capac- 
ity under public discussion. 



Theorem 4 An upper bound on the secret-key capacity is 



R+ =max/(x,s;yr|ye). 

Px\s 



(8) 



We note that the upper bound expression (O is similar to the 
upper bound expression in ^ except that we cannot min- 
imize over the joint-probability distribution in ([8]l. This is 
because the public discussion channel provides a mechanism 
for feedback and hence the capacity does depend on the joint 
distribution (not just the marginal distributions). The proof 
for the upper bound expression in Theorem |4] also signifi- 
cantly more elaborate as it accounts for public discussion. 

We note that if the channel additionally satisfies yr — > 
(x,s) — 7> ye then the upper and lower bounds in Theorem [3] 
and|2]coincide. In particular if p^^^ is the maximizing distri- 
bution in (O, we have that 

Rai,^>I{x,s;yr)-I{ye;yr) 

^i{ye,x,s;yr)- /(ye; yr) - /(yr; ye |x, s) 
= /(ye,x,s;yr)-/(ye;yr) 
= /(x,s;yr|ye) =/?|i3,. 

Since R^:^^^ < R'^:^^^., it follows that the two expressions must 
be equal. This is summarized in the result below. 

Theorem 5 The secret-key capacity with public discussion 
for a DMC channel that satisfies y^ — > (x, s) — ^ y^ is given by 



Qisc = max/(x,s;yr|ye). 

Px|s 



(9) 



3.3 Gaussian Case 

We now study the Gaussian special case under an average 
power constraint. The channel to the legitimate receiver and 
the eavesdropper is expressed as: 



yr 
ye 



^r 



(10) 



where Zr ~ ^(0,1) and Ze ^ ,yV{0,l + A) denote the ad- 
ditive white Gaussian nose and are assumed to be sampled 
independently. The state parameter s ~ c/K(0, Q) is also sam- 
pled i.i.d. at each time instance and is independent of both Zi 
and Ze. Furthermore, the channel input satisfies an average 
power constraint E [x^] < P. As the title indicates, we call 
this setup, secret sharing with dirty paper. 

Thus the parameter P denotes the signal-to-noise ra- 
tio, the parameter Q denotes the interference-to-noise-ratio, 
whereas A denotes the degradation level of the eavesdropper. 
We now provide lower and upper bounds on the secret-key 
capacity with and without public discussion. For simplicity 
in exposition we limit our analysis to the case when P > I. 

Proposition 2 Assuming that P > I, a lower bound on the 
secret-key agreement capacity is capacity is given by. 



4.1.2 Encoding 

• Given a state sequence s" the encoder selects a sequence 
u" randomly from the list of all possible sequences that 
are jointly typical with s". 

• At time i = l,2,...,n the encoder transmits symbol x,- 
generated by sampling the distribution Px\u,s{'WiT^i)- 



4.1.3 Secret-key generation 

• The decoder upon observing y" finds a sequence u" 
jointly typical with y". 

• Both encoder and the decoder declare the bin-index of u" 
to be the secret-key. 



R 



= — log 
2 ^ 



A(p+e+2p^) 



P+Q+l+A + 2p^/FQ 
where p < I is the largest value that satisfies 



1 



P + Q+l 



(11) 



(12) 



Proposition 3 In absence of public discussion, an upper 
bound on the secret-key capacity is given by. 



1 

2 log 



A(P + e + 2v^) 
P + Q+l+A + 2y/PQ 



(13) 



It can be readily verified that the upper and lower bounds 
coincide in several asymptotic regimes. 

Proposition 4 The upper and lower bounds on secret- 
capacity without public discussion satisfying the following 



yp > 0, R+ R < 
lim R+-R^=Q 



1 



lim R: 



R-=0 



(14) 
(15) 
(16) 



Proposition 5 In the presence of public discussion, the 
secret-key capacity is given by the following expression. 



R^ 



1 



■log 1 



(i+A)(p+e+2^) 



P + Q+l+A + 2y^PQ 



(17) 



4. WITHOUT PUBLIC DISCUSSION 



In this section we provide the coding scheme and the upper 
bound for the case when there is no public discussion. 

4.1 Proof of Theoremd] 

A sequence of length n code is described as follows. 



4.1.4 Secrecy Analysis 

We need to show that for the proposed encoder and decoder, 
the equivocation at the eavesdropper satisfies 



-H{K\y",)=I{u-yr)-I{u;y,)+o„{\), (18) 
« 



where o„{l) is a term that goes to zero as n — > °°. 
Accordingly note that 



^-H{K,u"\y:)~Ul{u"\y':,K) 

-H{u"\y':)--H{u"\y^,,K) 
n n 

-H{u"\y'l)-en 



where the last step follows from the fact that there are at-most 
2«(/(u;ye)-o„(l)) ggqiigfices in each bin and hence the eaves- 
dropper can decode the codeword u" given the key K. It re- 
mains to lower-bound the first conditional entropy term. 



-H{u"\y':) = -H{u") + -H{y':\u") - -H{y':) (19) 
n n n n 

= -H{u") + -H{y1 1 u" , s" ) - i//(y« ) + -I{s" ; y," | u" ) 
n n n n 

(20) 



We now appropriately bound each tenn in ( I20l l. First note 
that since the sequence u" is uniformly distributed among 
the set of all possible codeword sequences, it follows that 



4.1.1 Codebook Generation 

• Generate a total oy 2" '^^^"^ sequences. Each se- 
quence is sampled i.i.d. from a distribution Pu{')- 

• Select a rate R ~ I{u\y-[) —I{u;y^) — £„ and randomly 
partition the set sequences in the previous step into 2"^ 
bins so that there are 2"i'i"'yr)^^") sequences in each bin. 



-H{u") = -log2\'^\-£n 

n n 

= /(u;yr)-e„ 



(21) 



Next, given (u", s"), as verified below, the channel to the 



Figure 2: Secret-key agreement codebook for the dirty paper channel. The transmitter signal x" is selected so that u" = x" + s" 
is a sequence in the random codebook. The legitimate receiver can decode u" (with high probability) and map it to the secret- 
key. The eavesdropper's noise-uncertainity sphere includes are possible key values. Note that unlike the traditional dirty-paper 
code, the transmiter signal x" has a component along s". The achievable rate, does depend on the interference power and hence 
it is beneficial to amplify it using part of the transmit power. Also note that unUke a dirty -paper code we do not scale down s" 
before quantizing but use a = I. 



eavesdropper is memoryless: 

Py;!\u".s"iye\u",s") 

= I Py--lu^.s^y'{y:\M\s",x")p{x"\u",s"){x"W\s") 
n 

n 

= n E Pye\u,s.,x{yeMi-,Si,Xi)p^\^,{xi\ui,Si) 
n 

= Y\Pye\u..s{yeM,Si) 

i=\ 

The second step above follows from the fact that the channel 
is memoryless and the symbol x, at time i is generated as a 
function of {ui,Si). Hence we have that 

-H{y': I s" , u" ) = f H{ye., \ s" , u" ,y'-,') (22) 
n .^j 

= Y^H{y,M,Ui) (23) 

i=l 

Furthermore note that 

-Hi/:) < t"iy-)- (24) 



Finally, in order to lower bound the term /(s";y"| u") we 
let J to be a random variable which equals 1 if (s", u") are 
jointly typical. Note that Pr(y = 1) = 1 — o„(l). 

J/(s";ye"|u") = ^//(s"|u")-^//(s'"|u",ye") 

> ii/(s"|u",/= l)Pr(y= l)--//(s"|u",yf) 
n n 

> -His"\u",J=l)--H{s"\u",y':)-o„{l) 
n n 

>H{s\u)--His"\u'\y:)-o„il) (25) 
n 

>H{s\u)--yH{s,\ui,ye.i)-o„{l) (26) 



where ( 125b follows from the fact that s" is an i.i.d. sequence 
and hence conditioned on the fact that {s",u") is a pair of 
typical sequence there are 2"^(^l")^"""0 possible sequences 
s". 

Substituting (1211) . ( 123b , (124b and ( 126b in the lower 
bound ( 120b and using the fact that as n^ °°, the summation 




Figure 3: Bounds on the capacity of the "secret-sharing with dirty paper" channel with and without pubHc discussion. In the 
left figure, we plot the capacity as a function of SNR (dB) when 2=10 and A = 10. The upper-most curve is the capacity 
with public-discussion whereas the other two curves denote the upper and lower bounds without discussion. In the right figure 
we plot the capacity with public discussion as a function of A (in dB) when P = 10 dB and 2 = 10 dB as well as the upper 
and lower bounds without public discussion. 



converges to the mean values, 
\H{K\y:) 

= I{u-yr)+H{ye\u,s)-H{ye)+H{s\u)-H{s\u,ye)~o„{l) 
= /(u;yr) -/(ye;s|ty) - /(ye; u) +/(ye;s|u) -o„(l) 
= /(ty;yr)-/(ye;u)-o„(l) 

as required. 

4.2 Proof of Theorem|2] 

A sequence of length-n code satisfies: 

-H{K\y';) < e„ (27) 
n 

-H{K\y:)>-H(K)-E„ (28) 
n n 

where ( 127b follows from the Fano 's Lemma since the receiver 
is able to recover the secret-key K given y" and (I28l l is a 
consequence of the secrecy constraint. Furthermore, note 
that K — > (x",s") — > {y" ,y") holds as the encoder generates 
the secret key K. Thus we can bound the rate R = ^H{k) as 
below: 

nR<I{K-y'M)+2nen 

<I{K,s",x"-y:\y:)+2nen 
= /(s",x";y;'|y^')+2«e„ 

n 
1=1 

< n/(x,s;yr|ye)+2ne„ 

where the last step follows from the concavity of the condi- 
tional entropy term I{x, s;yr|ye) in the input distribution p^.s 
(see e.g., ^7?!/ ). 

Finally since the secret-key capacity only depends on the 
marginal distribution of the channel and not on the joint dis- 
tribution we can minimize over all joint distributions with 
fixed marginal distributions. 



5. WITH PUBLIC DISCUSSION 

In this section we provide the proofs of the coding theorem 
and the converse for the case when there is a public discus- 
sion channel allowed. 

5.1 Proof of TheoremlH 

Our coding scheme is closely related to the coding theorem 
for the channel model in ^ and emulates the generation 
of correlated source sequences. It consists of the following 
steps: 

• Fix a distribution p^^^. This induces a joint distribution 
Px.y,.y,.s- LetR = I{y,;x,s)-I{y,;ys)-e„ 

• Partition the set of all typical sequences y" into 
2n{H{yr\x,5)-o„{i)) [^-^^ Furthermore partition the collec- 
tion o/2"(^f-*'' '^'^'' sequences in each bin into further 2"^ 
sequences so that there are 2"^'^^''^'='^^^"^ sequences in 
each sub-bin. 

• Given symbol s; at time i, sample a symbol x; from the 
conditional distribution ;?x|s(') '^(nc/ transmit it over the 
channel. 

• The receiver upon observing y" transmits the bin index of 
this sequence over the channel. Using the bin index and 
the knowledge of (x" , s" ) the sender reproduces y". 

• Both the sender and the receiver declare the sub-bin in- 
dex of y" as the secret-key. 

Following the secrecy analysis in HS] it can be shown 
that this construction satisfies the secrecy constraint (|2|l and 
furthermore attains a rate of 

= /(yr;x,s) -/(yr;ye) +o„(l). 

5.2 Proof of Theoremll 

We now establish a corresponding upper bound on the secret- 
key capacity. 

First, using the fact that the receiver is able to recover 
the secret-key and the eavesdropper is subjected to a secrecy 



constraint (|2|l, we have that 



//(rc|y;,i/,4>*)<e„ 



I{K;y'^yM)<£n 



(29) 

(30) 
(31) 



Using the above relations and the fact that R ~ ^H(k), we 
note that 

nR < H{k) 

< I{K-yl\ v,4>^') -/()c;y,^4>^'^''^) + 2«e„ 
<I{k- y'' , y," , 4>*- , ^^•) - /( fc; y^" , <I>^ 'P*^) + 2ne„ 
<I{K;y^',v\y'^My)+2nen 

</(u,s";y;^u|y^,4>^¥*)+2ne„ (32) 
= I{u,s"-y:\v,y'^,<p',^')-I{u,5"-y':M,'i'')+2ne„ 
= /(u,s";\/,<I>'i-',1"i->) 

+ I{u,s";y^\y^M, + i.%+i\'^''-\'i'"-\v) 

-/(u,s";4>'i-',^'i-i) 

-/(a,s";y«,4>f,+i,^f^ + i|>I">-',<I>'>-i) 

= /( s" ; u lO'i - \ 1"i - 1 ) + £ F,,j + Grj - f F,j + Gej 

(33) 

where we have introduced 

Fr,j=I{u,s"-yr.j,ye.j\v,^'^-\w'^-\yt\yt') (34) 
Grj = I{u, s"; v/Jir'. <Pi;+^'\ f ¥'^-\yJ.yi) (35) 
iVj=/(u,s";y,j|0'^-i,V/'^-i,i/) (36) 

G,,,=/(o,s";</.j;^^W;:^'lf^"^r'^"^y^) (37) 



To complete the proof, it suffices to show that the follow- 
ing relations in (133b hold 



I{u,s"■v\^'^-\^>•^-^)^0 

Fr.j-Fe,j<I{Xj,Sfyrj\yej) 
G,j~GeJ<Q 



(38) 
(39) 
(40) 

To establish (1381 ) note that for < k < i\ — I we have that 
= O^, ( u, s" , ' ) and likewise 'I'^ = ( i/, O*^" ' ). Using 
which 

</(u4>„_i,s";^,1'„_i|4>'i-2,1'"-2) 
= /(u,s";i/,<I>'i-^>I"i-2) 

Continuing this process we have that 

/( u, s" ; u lO'i 1"i -')</( u, s" ; 1/) = 0, 

where the last relation follows from the fact that v is inde- 
pendent of {u,s"). 



In order to establish ( I39I ). we use ( 1341 ) and ( I36l l to get, 

^rj — Fe.j 

^I{u,s"; yr, , ye.j \v,r^-\xif'^-\ yt ' , y/- ' ) 

-I{u,s";y,,j\ij>'^-\w'^-',yt') 
= H{yr, , y..; \vA''-\w'''\ yt ' , y/" ' ) 



-H{y,-j,yej\v,0'j- 
-H{yejW'-\xif'r 
+ H{yeM'-\\ir''- 
= H{y,,j,ye.j\v,(j)'J-\ 
-H{yrj,ye.j\v,^'J' 
-H{y,,0'-\xi/'r 



-H{ye.jWi-\w'- 



,w'^-\yi-\yr\u,s'') 

,yt') 

,yt\u,s") 

w''-\yt\yt') 

,Vf'^-\yi-\yr\u,s",xj) 

.yf) 

,yi-\u,s",Xj) 



<H{y,-,j,y,,jW'-\x^y 
-//(y,j|(/)0-i,v/' 



(41) 



' \yi ^)~H(yr.j,yej\Xj,Sj) 

,yi-')+H{y,,\xj,Sj) (42) 
< H{yrj\ye,j) - H{y,-j\ye,j,Xj,Sj) ^ I{sj,Xj;y,-j\ye.j) (43) 

In the above steps (141b follows from the fact that Xj = 
Xj{u,s" , y/'j^^) and hence we can condition of xj i the sec- 
ond and fourth tenets. Furthermore since the channel is mem- 
oryless 

{yr.j,ye,j) [xj^Sj) -^{v,u, , sf w'''\yt\yt 

holds. 

It remains to establish (140b . Using (1351) and (137b we note 
that 

GrJ - Ge.j 

=/(u,s";v/;;^r',0;;;rV,f^"Sr'^"',y/,W) 

= H{u,s"\v,<^'^-\x^'^-\yi,yi) 

-//(u,s'>,(/)'^+'-',r'^+'-Sy/,yi) 

- H{u, s" W'-\ v'^-' , yi) +H{u, s" I f , w'^-' ' ' , yi) 
= /(u,s^^/,y/|(/)'V+'-^v/'^+l-^y^■) -7(0,5"; >/,y/|yi,0'^-S 

Since = 4>,v^|_i(u,s", -2) and Wij+i-i 

^iJ^^^l{v,yr —2) we have that 

I{u,s"-v,yiWi^^-\^lf'i^^-\yl) 
< I{u, s'\<j>,^^,_uv,y,\ v/,-,^._i v/'^>'-2,yi) 
= /(u,s";^,y/|0O+i-2,v/'^>i-2,y;) 

one/ continuing this process we have that 

/(u,s"; v,yJ\fi^'-\w'J^^-\yi)<Iiu,s"; v,yJ\yi 

as required. 

6. GAUSSIAN CASE 

In this section we develop the corresponding results for the 
Gaussian case. 



6.1 Proof of Prop. |2] 

The lower bound expression follows from Theorem [7] by 
choosing x ~ .yy{0,P) to be a Gaussian random variable 
independent of s and by choosing u = x + as. In this case, 

R = I{u-y,)-I{u\y,) 
= h{u\y^) - h{u\yr) 

Further evaluating each of the terms above with u = x + as, 
note that 



a'-Q + lap^PQ- 



{P + aQ+{\ + a)py/PQf 



h{u\y,) = 
and 

h{u\y,) = 

1, 2^ ^ iP + aQ + p(l + a)^/PQf 

-logl P + a^Q + lap y/PQ-^ ^ ^' 



P + Q + \+2y/PQ 



This yields that 



R 



■log 1 



1 



P+a^Q+2pa.JPQ , 

P + Q+l+2p^ 



— log , 

2 ^\P+Q 



l+A + 2py/FQ 



(44) 



Note that the first term in the expression above is maximized 
when a ~ I. As we show below, this choice is indeed feasible 
when P >\. In particular the constraint Q requires that 

h{u\s)>h{u\yr) 

^ilogP(l-p2)> 

Ij^ f PQ(,a - 1)^(1 - P^) + (f + a^Q) + 2pay/PQ) 
2 °^ V P + e+l+2pVPe 



Substituting a ^ I above we have that 
P{l-P^)>l- ^ 



P + Q+l+2py/PQ 



> 1 -■ 



1 



P + Q+l 

(45) 



as required. 

6.2 Proof of Prop, m 

We evaluate the upper bound in Theorem \2\for the choice 
Ze = Zi- + Zg, where zg ^ ^A^ (0, A) is independent of z^. 

/(s,x;yi.|ye) = h{y,\ye) - h{y,\y^,x,s) 
= h{yr\ye) - h{zr\z^ 

< i log f P+ e + 1 + 2,^ - i^±e±i±Mpl 

where we have used the fact that the conditional entropy 
/i(yr|ye) is maximized by a Gaussian distribution. The above 
expression gives (fTsT l. 



6.3 Proof of Proposition |5] 

Since the Gaussian model satisfies the condition in Theo- 
rem^ it suffices to evaluate C = /(x,s;yr|ye). 



^(^,s;yr|ye) =/j(yr|ye)-/!(yr|ye,x,s) 
^ h{y,\y^) - h{z,\z^) 



(46) 
(47) 



= liog2..fp+e+i+2v^- 

(48) 

which upon simplifying yields the desired expression. 
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